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Abstract 

The papers in this Special Issue were initially prepared for an EARLI 2013 Symposium 
that was designed to examine methodologies in use by researchers from two sister 
communities, Learning and Instruction and Learning Sciences. The four papers reflect a 
common ground in advances in conceptions of learning since the early days of the 
“cognitive revolution ” in the 1960s. This commentary shows the interdependence between 
advances in theory and advances in methodologies. Lour shifts in conceptions of learning 
are described. That these shifts are evident in the work of both communities suggests a 
blurring of the boundaries between the two. 
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The papers in this Special Issue were initially prepared for an EARLI 2013 Symposium that was 
designed to examine methodologies in use by researchers from two sister communities, Learning and 
Instruction and Learning Sciences. A main goal was to explore what these methodologies might reveal 
about underlying conceptions of learning and potential common ground across the two communities. Indeed, 
the four papers, taken together, reflect a common ground in advances in conceptions of learning since the 
early days of the “cognitive revolution” in the 1960s. The papers depict highly systematic, thoughtful, and 
rigorous approaches to studying learning as it is happening whether individually, in small groups or large; in 
classrooms, in workplaces, or in labs; face - to - face or virtually; in one moment in time or over extended 
periods. They also illustrate the interdependence between advances in theory and advances in methodologies. 

During the 30 year period from 1960 - 1990, the majority of studies of learning took place in one 
location: the laboratory; looked at individual cognition as a function of an operationally defined and 
restricted set of variables, in one time frame with occasional return visits. Although some researchers were 
examining learning in the context of tasks students might be asked to do in school, many of the “learning” 
situations were set up as experiments that were highly constrained to maintain experimental control over 
“extraneous” variables; as well the tasks were frequently “toy” problems that had little relevance to 
classrooms or other contexts outside of cognitive theory and the academic settings in which the research was 
being conducted. Consequently, it was difficult to see how findings from the lab could possibly have 
relevance to everyday learning. Emphases were on manipulating characteristics of the materials, the task, or 
both and observing how people “solved” the tasks and their success at doing so with some emphasis on 
understanding how they had completed the tasks. Some of the findings emerging from that research shaped 
instructional studies in which people were instructed to summarize sections of text, underline main ideas, 
break down tasks into subtasks before solving them, group words based on taxonomic categories to improve 
memory, and similar heuristics. Published studies of this sort attest to the success of these approaches in 
building a cognitive theory of learning and problem solving that superceded extant behaviourist/empiricist 
views (cf. Greeno, Collins, & Resnick, 1996). There was, however, little uptake of these theories by 
educational practitioners. 

In addition to changes in the conceptualization of learning and the kinds of questions being asked 
about learning, data analytic methodologies have come a long way since the 1960s. Some of the (older) 
readers of this article will remember the days of doing ANOVAs by hand, with “advances” marked by 
programmable Wang calculators, and main frame programs that automated the process. Of course, you had 
to batch process jobs, submitting stacks of punch cards containing the data (all the time living in fear that 
you would drop the deck and have to start again making sure they were in the right order) and then wait for 
the print out. Turnaround varied from 10 minutes to 24 hours. Over the past 30 years there have been huge 
advances in the technologies and data analytic applications available on devices that are small enough to 
carry around the way we once carried pads of paper and notebooks (not electronic ones). These technologies 
have expanded the ways we dare to think about analyzing our data, enabled us to collect and make sense of 
new forms of data, and automated or semi-automated analysis methods that we used to do solely by hand. 

Currently, the Learning Sciences and Learning and Instruction communities operate with theoretical 
frameworks on learning that reflect more complex views of learning in four major ways. We now understand 
and attempt to study learning 

1. in multiple and iteratively designed environments 

2. over multiple time scales 

3. occurring in social groups of multiple and collaborating individuals 
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4. with effects evident at multiple levels ranging from behavioural to neural. 


As a set, the papers in this Special Issue reflect these shifts in conceptions of learning and its 
investigation and provide us with methodological tools that enable us to rigorously investigate learning 
processes and outcomes despite the greater complexity of doing so. There is evidence of one or more of 
these shifts in conceptions of learning among researchers who identify with Learning Sciences, as well as 
among those who identify with Learning and Instruction, suggesting something of a convergence, or at least 
a blurring of the boundaries of the two communities. 


1. Learning in Multiple and Iteratively Designed Environments 

Design-based research marked a pivotal shift in perspectives on learning and its study in classrooms. 
Svihla (this issue) provided an excellent description of the goals of this research approach. Up until the time 
that this approach to research on learning was introduced in the early 90s by Ann Brown (1992) and Allan 
Collins (1992), educational research in schools typically took the form of relatively short-term experiments 
that involved comparisons of the effects of different methods or materials on various cognitive skills. The 
studies were usually conducted by the researchers. Teachers “cooperated” with the researchers in terms of 
providing access to their students for the duration of the study but were otherwise minimally involved in 
contributing to the instructional design or the materials. A major goal of this research was ascertaining which 
instructional methods were better than others for achieving largely cognitive objectives such as more 
accurate mathematics performance, better memory for new vocabulary, and better comprehension of text. 
Accordingly, assessments were designed to measure changes in students’ performance as a function of 
having participated in the study either in the “experimental treatment” or the “control” group. Along with 
these types of cognitive studies there were similarly designed studies that examined the impact on 
individuals of having worked in cooperative groups (e.g., Johnson & Johnson, 1999; see for review Webb & 
Palincsar, 1996). 

As Svihla described, the goals of DBR reflected a fundamental shift to an emphasis on studying 
learning processes in situ as both social and interactional (Collins, 1992). Learning processes were studied in 
the context of designed learning environments developed through collaborations of researchers and 
practitioners and based on principles that constituted a learning theory. Enactments of designs were objects 
of study for purposes of understanding how, with the understandings that emerged from close study of, and 
reflection on, the interactions and student work informing iterative refinement of the learning theory 
principles, and designs. Svihla does an excellent job of depicting the ways in which DBR has developed 
since its initial introduction. Suffice to say it made apparent the need for methodologies to capture processes 
occurring over multiple time scales and among individuals in social configurations. 


2. Learning processes over multiple time scales 

Complex views of learning make it clear that processes occur over time, with different learning 
processes occurring at different time scales. Molenaar (this issue) provided an excellent rationale for the 
need for temporal analysis methods. She described a variety of the issues involved in shifting from a focus 
on whether a particular construct has been learned or not to a focus on how that construct is learned, what 
that learning looks like at different time scales, and indeed what constructs are conceptualized as emerging 
over longer versus shorter time frames (cf. Lemke, 2000). She referenced a variety of constructs that we now 
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think of as emerging over events and across time but that used to be thought of as personality traits (e.g., 
motivation, persistence). She discussed various computational tools that can aid in segmenting, coding, and 
relating different time scales. These methodologies are critical to doing the analyses needed to understand 
learning over different time scales. 

A variety of issues face us as individual researchers and as a community as we apply methodologies 
for temporal analysis: What units of time are appropriate for particular constructs of interest, especially when 
multiple time scales operate in parallel? How do we determine the time scale most appropriate for tracing the 
emergence of a construct over time? Or alternatively, how do we capture the interrelationships between 
events occurring over time but at different time scales? Of potentially many patterns of events that might be 
extracted by pattern detection software, how do we determine which are psychologically meaningful and at 
what scale of time they are meaningful? 

Equally necessary are new forms of representation that can assist us in conveying our findings to the 
broader community. Molenaar (this issue) presented one form of representation. Figure 1 illustrates a 
different form of representation in which we plotted the discourse moves of three students comprising a 
small group engaged in a science investigation (Radinsky, Goldman, Doherty, & Ping, 2010). This particular 
figure shows the moves for the first day of the investigation. We plotted similar representations for each day 
and then used the graphs to identify regions where there were clusters of moves across the three students that 
suggested there were interesting dialogic discourses occurring. We then “dove” into these segments of the 
discourse to determine the character of the “argumentation” in which the students were engaged and whether 
the claims and evidence being offered were similar or different later in the investigation versus earlier. As 
well, we considered how participation and roles in the discourse revealed dimensions of identity and 
positioning with respect to disciplinary competence. This form of representation was a useful analytic tool 
and with some refinement might be a useful way to represent the time course of argument development 
(Radinsky, et al., 2010). 

Molenaar highlighted the need to conceptualize different dimensions of time in order to define 
important temporal characteristics. She cited papers by Bloome, et al. (2009) and Lemke (2000) as informing 
this discussion. In addition, Bahktin’s (1981) framings of time in relation to discourse, meaning, and learning 
will be a useful resource. We also need to (re)connect learning and development. Indeed, the move from the 
more traditional educational research paradigms to DBR and related learning sciences methodologies creates 
a convergence between research on development and research on learning. That is, one distinction between 
development and learning had traditionally been the time frame over which phenomena of interest emerged. 
Those that occurred over multiple years were called developmental, e.g., oral language; those over minutes 
or hours, learning, e.g., declarative knowledge such as “c - a -1” spells cat; or propositions such as EARL1 
is a professional research organization. Arguably, a second distinction was whether the phenomenon 
emerged with or without formal instruction, the latter being deemed developmental phenomena and the 
former learning. For example, children develop oral language but learn to read with explicit instruction. 
Because developmental psychologists have long been concerned with the study of change over time, they 
have developed techniques that examine change over relatively longer periods of times such as growth 
analysis (Willett, 1989), as well as techniques that look at moment - to - moment change, such as sequential 
analysis (Bakeman & Quera, 1995) and microgenetic analysis (e.g., Kuhn, 1995; Siegler & Stern, 1998). 
These methods are rich resources for examining learning over multiple time scales. 
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3. Learning in social groups of multiple and collaborating individuals 

A core assumption of the Learning Sciences is that learning is social and interactional and takes 
place through situated activity (Brown, Collins, & Duguid, 1989). Collins, Brown, and Newman (1989) 
labeled the approach cognitive apprenticeship, reflecting the importance of observing the habits of mind as 
well as the actions of the more knowledgeable others in the community (cf. Vygotsky, 1978). Hence, 
discourse about activity and interaction with others engaged in the activity became a central focus for 
understanding learning. Researchers with intellectual roots in a variety of disciplines have long relied on 
discourse among participants in a joint activity as a window into knowledge building processes of groups as 
well as individuals, and, along with gestures, into processes of learning through joint activity (Gee, 1992; 
Goodwin, 1994; Hutchins, 1995; Lave & Wenger, 1991; Sawyer, 2006; Scardamalia & Bereiter, 1991; 
Schegloff, 1991, 2007). Video and audio recordings have typically provided the raw data and various types 
of very time-consuming and intensive qualitative analyses have been used by researchers to provide evidence 
for claims about learning outcomes and processes. Frequently and understandably given the labor-intensive 
nature of these analyses, the evidence provided in any one empirical report has tended to be based on 
relatively small data sets or corpora. 

Many computer-supported collaborative learning (CSCL) environments make available written 
traces of learning interactions that can also be mined to understand learning processes and outcomes for 
individuals and for groups. Although initially these were also analyzed by humans using processes similar to 
those used for coding discourse that was transcribed from video and audio recordings, a number of 
computer-assisted methods have been developed that make the work of coding less time-consuming. 
Stegmann (this issue) argued that to understand the mechanisms that produced enhanced learning outcomes 
in CSCL, three hypotheses needed to be tested. These have been conceptualized as a “triangle of 
hypotheses:” “(a) instructional/technological support facilitates learning activities; (b) facilitated learning 
activities have positive effects on learning outcomes; and (c) mediated by learning activities, 
instructional/technological support has a positive effect on learning outcomes.” (Stegmann, this issue, p. #, 
citing Wecker, Stegmann & Fischer, 2013; Fig. 1). It could be argued that these three hypotheses can be 
thought of as constituting an activity system (Engestrom, 1987, 2001) in which tools (technology support), 
activities, and performances of individuals and groups exist in interaction with one another and over time. 
Stegman and colleagues argue that conceptualizations other than experimental designs are needed to 
establish relationships between learning tools, activities, and outcomes. They propose the use of nomological 
nets to ensure that direct and mediating relationships between the tools and outcomes can be tested. 
Nomological nets specify what constructs are indexed to which observables over what time frames. As well 
interrelationships among constructs are specified. Empirical evidence derived from collaborative activities 
constitute input to revisions and refinements of theoretically grounded nomological nets. These revisions 
may reflect mediational variables that become evident through indepth analyses of the discourse, of changes 
in the interactions and discourse over time, and at different ’’units” of analysis (e.g., individual, dyad, small 
groups, entire activity system). 

Stegmann (this issue) argued that the indepth analyses required to “test” initially specified 
nomological nets should take advantage of statistical techniques designed to detect patterns of interactions 
as they occur over time. These techniques require some form of quantified information; therefore, 
qualitative analyses need to be quantified. Fortunately, there are a number of computational algorithms that 
can assist researchers in doing so. Importantly, these systems assist researchers in parsing the input as well as 
counting instances of particular codes and discovering repeating sequences of codes. The construct 
specification required by nomological nets is one way of ensuring that codes, sequences of codes, and 
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recurring patterns relate to theoretically meaningful constructs. Thus nomological nets can assist researchers 
in determining whether “discovered” patterns have psychological validity and practical utility. 

Construct specification in nets can also assist with an additional “sticky wicket” in efficient yet 
automated detection of meaningful patterns of interactions. Essentially, over what time frame are patterns of 
interaction to be detected and at what levels? That is, if a pattern is detected in a series of successive turns 
does that pattern then become a “unit” that can act as input to a subsequent pattern analysis effort? How are 
such patterns related to constructs in the net? One might envision a series of intermediate level patterns being 
inferred from turn by turn coding. These intermediate levels (essentially patterned sequences of turns) are the 
units upon which further pattern detection analyses are conducted. Determining and optimizing appropriate 
time scales over which patterns of code sequences are constituted depends on understanding the intentions 
and assumptions of specific designed learning environments, particularly what and when specific processes 
are expected; why, and how they support expected outcomes. Although patterns and sequences can be 
detected automatically it will take human interpretive lenses and socio-cognitive theories to specify the 
constructs these index and their meaningfulness in the context of learning. The issue of levels is relevant not 
only to pattern detection but to learning in general, as reflected in the fourth aspect of a more complex view 
of learning. 


4. The effects of learning are evident at multiple levels ranging from behavioural to neural. 

Learning is “visible” at different levels. De Smedt (this issue) is to be applauded for emphasizing the 
need for alignment between the level and topical focus of research questions and the methods selected to 
investigate the questions. He pointed out that if the research question is targeted at the macrolevel, 
behavioral methods would be most appropriate. Cognitive neuroscience methods become appropriate for 
research questions focused on microlevel processes. The two most common cognitive neuroscience methods 
are electroencephalography (EEG) and functional Magnetic Resonance Imaging (fMRI). EEG methods 
provide temporal information about when particular processes are taking place and fMRI methods provide 
spatial information about where in the brain processes are taking place. Cognitive theories provide needed 
links between behavioral and neural levels. 

Furthermore, echoing the point made above - that socio - cognitive theory needs to guide the 
interpretation of patterns in interactions, De Smedt (this issue) called for detailed cognitive theory of learning 
phenomena to provide needed links between behavioral and neural levels. He cited the cognitive theories and 
the behavioral data on which they are based as critical for interpretations of the information that results from 
the application of cognitive neuroscience methods. To demonstrate his claims, De Smedt (this issue) 
illustrated three ways in which cognitive neuroscience methods elucidate mathematical instruction. These are 
convincing demonstrations of the value added of obtaining data on the same phenomenon at multiple levels 
and coordinating findings across levels. Predictions can be pursued across levels by postulating what should 
be the case at one level based on manifestations at another level. 

The value added of using multiple methods to examine learning at multiple levels is not restricted to 
mathematics. For example, in the area of language acquisition, researchers have used EEG methods to 
establish predictive relationships between phonemic and word-level development. Specifically, infants below 
six months of age are sensitive to phonetic contrasts in all languages; between six and 10 months, a 
perceptual narrowing process occurs that results in sensitivity to only those phonetic contrasts that matter in 
their native language. EEG methods have established that better neural discrimination of native language 
phonetic contrasts is associated with faster vocabulary development (Kuhl & Rivera-Gaxiola, 2008). Kuhl 
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and colleagues have also used neural activation patterns to determine that the perceptual narrowing process 
occurs several months later for infants reared in two-language homes compared to those reared in 
monolingual homes. Finally, neural indicators of phoneme learning demonstrate that social interaction plays 
a critical role in language acquisition (Kuhl, 2007). In each of these cases, evidence from the neural level 
provides measures of far greater precision than could be obtained behaviorally. 


5. Summary and Challenges 

More complex views of learning require more complex methodologies for addressing key questions 
about learning and the conditions that support it, including explicit instruction. The four papers in this special 
issue illustrate methodologies that assist with capturing the iterative design-based research process, learning 
processes and outcomes that occur at different time scales and levels, and make possible the formulation and 
testing of hypotheses that relate different levels to one another. The papers present examples of ways in 
which these methodologies are augmenting the knowledge base for understanding learning as it occurs 
across individuals as well as within individuals. As such they make valuable contributions to the 
field.Moving forward, there are a number of areas that need attention in terms of further theoretical and 
methodological development. Briefly, more emphasis needs to be devoted to formative assessment that 
provides opportunities to better facilitate instructional processes and outcomes. This includes the design and 
testing of tools for capturing learning interactions that are classroom, teacher, and student friendly. Such 
tools would enable students and teachers to reflect on their learning processes as well as outcomes at much 
finer levels of detail than is currently feasible. Ideally, researchers would develop and test various 
technology-based tools for accomplishing these goals and would then engage in “user testing” of tools that 
travel outside research labs and into the hands of teachers and learners. A type of tool that would be helpful 
in this process is one that enables visualizations of the ebb and flow of learning processes across people and 
across time. 

Finally, the Learning Sciences community has tended to design within specific disciplines and fields; 
the Learning and Instruction community has tended to test principles and variables thought of as general 
across all learning situations. Neither perspective has as yet come to grips with the tension between 
generalist and discipline-specific views of learning nor the limitations of each view. What is needed are 
studies that 1) embrace a disciplinary perspective but that also situate that discipline in the context of 
epistemological orientations and inquiry methods that have been adopted and developed within other 
disciplinary communities; and 2) examine the “fit” of principles, constructs, and explanatory mechanisms 
suggested by cognitive, developmental, and social psychological research to learning phenomena observed in 
designed learning environments. Studies of the first type will advance our understanding of the general and 
idiosyncratic aspects of learning in different disciplines. Studies of the second type will advance our 
understanding of explanatory mechanisms that have traction across a wide versus narrow band of learners 
and situations of learning. There are also aspects of learning processes and outcomes that need far more 
systematic and sustained research over shorter and longer time scales. Specifically, we need to conduct 
systematic research on relationships among persistence, engagement, identity, learning processes and 
outcomes, within and across formal and informal contexts of learning. Some of this research is currently 
being conducted; more of it needs to be conducted. The methodologies discussed in these papers can be 
synergistic with respect to tackling these challenges. 
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Keypoints 

Contemporary views of learning depart from solely “in the head” views of learning. 

9- Learning occurs in multiple and iteratively designed environments over multiple time scales. 

& Learning occurs in social groups of multiple and collaborating individuals with effects evident at 

multiple levels ranging from behavioral to neural. 

9 New methodologies are needed to capture the processes and outcomes of this complex perspective on 
learning. 
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Figure 1. Representation of the discourse moves of three students engaged in a science inquiry task, 
(cf. Radinsky, J. L., Goldman, S. R. Doherty, R. & Ping, R. (2010, June). Small Group 
Argumentation with Visual Data: Negotiating What Is Seen and What It Means. Paper presented at 
the annual meeting of the American Educational Research Association, Denver, CO.) 
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